Method and apparatus for providing an intelligent response

ABSTRACT

A method of providing an intelligent response on an electronic device and an apparatus therefor are provided. The method includes receiving a command from a user of the electronic device, analyzing the command, generating, based on the analyzed command, at least one intelligent response corresponding to the command and an actual response corresponding to the command, the at least one intelligent response including at least one of a sound effect associated with the command, a vibration effect associated with the command, or a visual effect associated with the command, and outputting the at least one intelligent response and the actual response by the electronic device.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. § 119of an Indian patent application number 201841019725, filed on May 25,2018, in the Indian Patent Office, and of an Indian patent applicationnumber 201841019725, filed on May 22, 2019, in the Indian Patent Office,the disclosure of each of which is incorporated by reference herein inits entirety.

BACKGROUND 1. Field

The disclosure relates to user experiences between a user and anelectronic device of the user. More particularly, the disclosure relatesto a method and an apparatus for managing a fake response to the user bythe electronic device.

2. Description of Related Art

Generally, a user executes one or more commands on an applicationinstalled in an electronic device. The command can be, for example, butnot limited to, a voice command, a text input, a touch input or thelike. In an example, the user of the electronic device provides thevoice command to an application (e.g., smart assistant application,virtual assistance application or the like) to perform a certain action.The command is used to control some devices or to perform certainactions via some applications (e.g. setting alarm, setting a reminderetc.). After an execution of the command, the user may receive afeedback (or an execution report) through a user interface. Generally,the feedbacks from a device are notifications or pop-ups.

Thus, it is desired to address the above mentioned disadvantages orother shortcomings or at least provide a useful alternative.

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providea method of providing an intelligent response on an electronic device.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

In accordance with an aspect of the disclosure, a method of providing anintelligent response on an electronic device is provided. The methodincludes receiving a command from a user of the electronic device,analyzing the command, generating, based on the analyzed command, atleast one intelligent response corresponding to the command and anactual response corresponding to the command, the at least oneintelligent response including at least one of a sound effect associatedwith the command, a vibration effect associated with the command, or avisual effect associated with the command, and outputting the at leastone intelligent response and the actual response by the electronicdevice.

In an embodiment, the command includes at least one of a voice command,a text command, a touch input command or an image input captured by acapturing device.

In an embodiment, the outputting the at least one intelligent responseand the actual response includes outputting the at least one intelligentresponse and the actual response in a temporal order or simultaneously.

In an embodiment, the generating of the at least one intelligentresponse includes transmitting, to a server, a request of the at leastone intelligent response and the analyzed command, and receiving the atleast one intelligent response from the server.

In an embodiment, the actual response corresponding to the commandcomprises a control signal to be transmitted to at least one remotedevice for controlling, in response to the command, the at least oneremote device.

In an embodiment, the generating of the at least one intelligentresponse corresponding to the content of the command comprisesgenerating the at least one intelligent response based on the analyzedcontent of the command and information associated with the command, theinformation being obtained from an application installed in theelectronic device or a server connected to the electronic device.

In an embodiment, the visual effect associated with the command includesa visual effect of changing the brightness of a display of theelectronic device.

In an embodiment, the moving image associated with the command comprisesmoving images depicting the command.

In an embodiment, the analyzing of the command comprises obtaininginformation associated with the command from a database via a network.

In an embodiment, the visual effect comprises displaying a still or amoving image associated with the command.

In an embodiment, the generating of the at least one intelligentresponse to the command includes searching a network for first datarequired to generate the at least one intelligent response andgenerating the at least one intelligent response based on the firstdata.

In an embodiment, the generating of the actual response includessearching the network for second data required to generate the actualresponse and generating the actual response based on the second data.

In accordance with another aspect of the disclosure, an apparatus forproviding an intelligent response is provided. The apparatus includes adisplay, a speaker, at least one processor configured to receive acommand from a user, analyze the command, generate, based on theanalyzed command, at least one intelligent response corresponding to thecommand and an actual response corresponding to the command, the atleast one intelligent response including at least one of a sound effectassociated with the command outputted by the speaker, a vibration effectassociated with the command, or a visual effect associated with thecommand, and output the at least one intelligent response and the actualresponse.

In an embodiment, the at least one processor is configured to transmit,to a server, a request of the at least one intelligent response and theanalyzed command, and receive the at least one intelligent responsegenerated by the server.

In an embodiment, the visual effect associated with the commandcomprises a visual effect of changing the brightness of the display ofthe electronic device.

In an embodiment, the at least one processor is further configured totransmit, to at least one illumination device, a signal to change thebrightness of the at least one illumination device proportional to thevisual effect of changing the brightness of the display of theapparatus.

In an embodiment, the at least one processor is further configured tosearch a network for first data required to generate the at least oneintelligent response and generate the intelligent response based on thefirst data.

In an embodiment, the at least one processor is further configured tosearch the network for second data required to generate the actualresponse and generate the actual response based on the second data.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a bock diagram of an electronic device 100 for providing anintelligent response, according to an embodiment of the disclosure;

FIG. 2 is a block diagram of the server 200 for the intelligentresponse, according to an embodiment of the disclosure;

FIG. 3 illustrates various elements in the processor 110 of theelectronic device 100, according to an embodiment of the disclosure;

FIG. 4 illustrates a block diagram for providing an intelligent responseto a command, according to an embodiment of the disclosure;

FIG. 5 is an overview of a system providing an intelligent response to acommand, according to an embodiment of the disclosure;

FIG. 6 is a flowchart illustrating a method of providing an intelligentresponse, according to an embodiment of the disclosure;

FIG. 7 is a flowchart illustrating another method of generating anintelligent response by the server, according to an embodiment of thedisclosure;

FIG. 8 illustrates an example scenario providing an intelligent responseto an inquiry, according to an embodiment of the disclosure;

FIG. 9 illustrates an example scenario of providing an intelligentresponse with a sound effect, according to an embodiment of thedisclosure;

FIG. 10 illustrates an example scenario of controlling light operations,according to an embodiment of the disclosure;

FIG. 11 illustrates an example illustration in which an intelligentresponse associated with schedules is provided, according to anembodiment of the disclosure;

FIG. 12 illustrates an example scenario in which an intelligent responseassociated with a door is provided, according to an embodiment of thedisclosure;

FIG. 13 illustrates an example scenario in which an intelligent responseassociated with weather is provided, according to an embodiment of thedisclosure;

FIG. 14 illustrates an example scenario in which an intelligent responseassociated with sports event is provided, according to an embodiment ofthe disclosure;

FIG. 15 illustrates an example scenario in which an intelligent responseassociated with light on/off is provided, according to an embodiment ofthe disclosure;

FIG. 16 illustrates an example scenario in which an intelligent responseassociated with fan speed is provided, according to an embodiment of thedisclosure; and

FIG. 17 illustrates an example scenario in which an intelligent responseassociated with playing a song is provided, according to an embodimentof the disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein may be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of thedisclosure is provided for illustration purpose only and not for thepurpose of limiting the disclosure as defined by the appended claims andtheir equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

As used herein, the terms “1st” or “first” and “2nd” or “second” may usecorresponding components regardless of importance or order and are usedto distinguish one component from another without limiting thecomponents.

The accompanying drawings are used to help easily understand varioustechnical features and it should be understood that the embodimentspresented herein are not limited by the accompanying drawings. As such,the disclosure should be construed to extend to any alterations,equivalents and substitutes in addition to those which are particularlyset out in the accompanying drawings. Although the terms first, second,etc. may be used herein to describe various elements, these elementsshould not be limited by these terms. These terms are generally onlyused to distinguish one element from another.

Accordingly, the embodiments herein disclose a system for providing anintelligent response. The system includes a server communicated with anelectronic device. The electronic device is configured to receive acommand from a user and send the command to the server. The server isconfigured to receive the command from the electronic device anddynamically create a response to the command, wherein the feedbackimitates the content or the characteristics of the command received fromthe electronic device. The server is configured to transmit the feedbackcorresponding to the command to the electronic device while theelectronic device is executing the command or once the electronic deviceexecuted the command. Further, the electronic device is configured toreceive the response to the command and provide the response to the userwhile executing the command or once the command is executed. Further,the electronic device is configured to provide a surreal or anintelligent response for the command once the command is executed. In anembodiment, the surreal or the intelligent response corresponds tocreate at least one of an imaginary response, a virtual response, and anaugmentable virtual effect for the command.

Unlike methods and systems of the related art, the proposed system canbe used intelligently augments one or more special effects (e.g., audioeffect, video effect, vibration effect, or the like) while providing anintelligent response to a current command or an event occurrence throughthe electronic device. These special effects are produced with an actualcommand execution or the event occurrence. The intelligent response mayincrease the user experience while providing the actual commandexecution or the event occurrence.

The system intelligently associates special effects to generate acompletely imaginary situation that mingles certain effects associatedwith the user input or an event with the action being performed. Thesystem intelligently provides a more real-world like executionexperience corresponding to the command intended by a user. Whilegenerating certain effects, the system intelligently associatesplurality of attributes (e.g., capability of the electronic device, alocation of the electronic device or the like) of involved underlyingelements of entities related to various use case (e.g., a location ofIoT illumination devices, a number of the illumination devices in thatlocation, brightness of the illumination device, etc. The effects areproduced based on capabilities of the electronic device with which theuser is interacting. The effects can be, but not limited to, an audioeffect, a visual effect, a physical (e.g., vibration) effect, or acombination thereof. Before producing the effects, the systemintelligently checks if providing the effects makes sense to the userand does not annoy the user with redundant or unnecessary effects. Forexample, the user provides the command of “virtual assistant! Close thegarage door”. In response to the voice command, the electronic devicedetermines whether the user is located close to the garage. In order tocontrol the degree of the effects to be provided, the electronic devicemay have or obtain, with a GPS module included in the electronic device,the location information of the user and the garage location which isassociated with the actual command. If the user is determined to belocated close to the garage, the electronic device generates thebackground sound of closing the garage door in a less volume. If theuser is relatively away from the garage, the electronic device generatesthe background sound of closing the garage door in a large volume.

The system can be used to intelligently augment or combine one or morespecial effects while providing the feedback or the intelligent responseabout a command input by the user or an event occurrence through theelectronic device.

Referring now to the drawings, and more particularly to FIGS. 1 through17, where similar reference characters denote corresponding featuresconsistently throughout the figures, there are shown preferredembodiments.

Throughout the specification, the terms of “feedback” and “surrealresponse” may be used interchangeably with the terms of “actualresponse” and “intelligent response”, respectively.

FIG. 1 is a bock diagram of an electronic device 100 for providing anintelligent response, according to an embodiment of the disclosure.

Referring to FIG. 1, the electronic device 100 can be, for example, butnot limited to a cellular phone, a smart phone, a personal digitalassistant (PDA), a tablet computer, a laptop computer, a music player, avideo player, an Internet of things (IoT) device or the like. Theintelligent response can be, for example, but not limited to a virtualresponse, an imaginary response, an augmentable virtual effect or thelike.

In an embodiment, the electronic device 100 may include a processor 110,a communicator 120, a memory 130, and a display 140. The processor 110may be coupled with the memory 130, the communicator 120 (e.g., atransceiver) and the display 140.

In an embodiment, the processor 110 is configured to receive a commandfrom a user. The command can be, for example, but not limited to a voicecommand, a text command, a physical command such as swinging theelectronic device, an IoT command, a touch input or the like. Theprocessor 110 may be able to handle any mode of command. After receivingthe command, the processor 110 may be configured to parse and analyzethe content or the intent included in the command.

Based on the analysis result, the processor 110 may determine andgenerate an actual response and an intelligent response specific to thecommand. The actual response to the command may imitate the content orthe characteristics of the command received from the user. For example,if the command is “Volume up!”, then the actual response may be anaction of turning up the volume or a control signal to control a speakerof the electronic device 100 or a speaker of a remote device to turn upthe volume.

In an embodiment, the intelligent response imitating the action of thecommand may be determined by determining the characteristics of thecommand, wherein the characteristics of the command indicates a topicand/or an action to be performed by an application (e.g., virtualassistant application, an IoT application or the like), determining atleast one of capabilities of the electronic device 100. The processor110 may generate the intelligent response to the command based on the atleast one of the characteristics of the command or the capabilities ofthe electronic device 100.

Further, the processor 110 may provide the intelligent response specificto the command while executing the actual response to the command orbefore or after the actual response is executed. When the command isexecuted, the processor 110 may provide the intelligent responsecorresponding to the command. The intelligent response to the command,can be, for example, but not limited to an audio effect, a visualeffect, a physical effect, an audio-visual effect, or anaudio-visual-physical effect.

The at least one of the audio effect, the visual effect, the physicaleffect, the audio-visual effect, and the audio-visual-physical effectare generated based on resource detail and an effect property. The audioeffect, the visual effect, the physical effect, the audio-visual effect,and the audio-visual-physical effect is provided by combining at leasttwo of the foregoing effects.

In an embodiment, the processor 110 may obtain the content of thecommand and parse the content of the command to determine whatcorresponding action (control) and/or effects will be appropriate forthe command. Further, the processor 110 may determine a command type.That is, the processor 110 may determine whether the command is a voicecommand, a text command, a touch input command or an image inputcaptured by an image capturing device. The processor 110 may obtain avoice command execution metadata, a visual command execution metadata, aphysical command execution metadata based on the command type. Theprocessor 110 may dynamically determine an actual and intelligentresponse imitating the action or the content of the command based on thevoice command execution metadata, the visual command execution metadata,and the physical command execution metadata.

The processor 110 may execute instructions stored in the memory 130 andperform various processes. The communicator 120 is configured forcommunicating internally between internal hardware components and withexternal devices via one or more networks. In an embodiment, thecommunicator 120 is configured for communicating with the server (notshown).

Further, the memory 130 stores and updates a target knowledgebase basedon the command executed in the electronic device 100. The targetknowledgebase contains details about a command output and keeps updatingthe command output over the period of time. Further, the memory 130stores the intelligent response.

Further, the memory 130 stores information about of all kinds ofintelligent responses. The memory 130 maintains information aboutresources (e.g., sound files, pattern files or the like) applicable tothe intelligent response. The property of each the intelligent responseis maintained in various formats (e.g., json format or the like). In anexample, in case of visual effects, the visual effects may containinformation such as an address of a code to be executed while creatingthe visual effect. The address of a code could be a local addressreference or reference to some remote cloud location. In anotherexample, during visual scene creation operations, the code informationis used to generate the parameters related to visual components (i.e.,video frames). While playing the scene, an effect composer (not shown)creates the visual effects using a display sub-system of the electronicdevice 100 on which the surreal response is provided/played.

The memory 130 stores instructions to be executed by the processor 110.The memory 130 may include non-volatile storage memory. Examples of suchnon-volatile storage elements may include magnetic hard discs, opticaldiscs, floppy discs, flash memories, or forms of electricallyprogrammable memories (EPROM) or electrically erasable and programmable(EEPROM) memories. In addition, the memory 130 may, in some examples, beconsidered a non-transitory storage medium. The term “non-transitory”may indicate that the storage medium is not embodied in a carrier waveor a propagated signal. However, the term “non-transitory” should not beinterpreted that the memory 130 is non-movable. In some examples, thememory 130 can be configured to store larger amounts of information thanthe memory. In certain examples, a non-transitory storage medium maystore data that can, over time, change (e.g., in random access memory(RAM) or cache).

Although FIG. 1 shows various hardware components of the electronicdevice 100 but it is to be understood that other embodiments are notlimited thereon. In other embodiments, the electronic device 100 mayinclude less or more components. Further, the labels or names of thecomponents are used only for illustrative purpose and does not limit thescope of the disclosure. One or more components can be combined togetherto perform same or substantially similar function to provide the surrealresponse in the electronic device 100.

FIG. 2 is a block diagram of the server 200 for the intelligentresponse, according to an embodiment of the disclosure.

Referring to FIG. 2, the server 200 may include a processor 210,communicator 220 (e.g., a transceiver), and a memory 230. The processor210 may be coupled to the memory 230 and/or the communicator 220.

In an embodiment, when the electronic device 100 is not capable ofprocessing the command input by a user for providing an intelligentresponse due to the resource limitation, the electronic device 100 mayconnect to the server 200 for obtaining the intelligent response. Theprocessor 210 of the server 200 may receive the command from theelectronic device 100 via the communicator 220. After receiving thecommand from the electronic device 100, the processor 210 is configuredto generate the feedback specific to the command. The feedback specificto the command imitates the content or the characteristics of thecommand as described referring to FIG. 1.

The actual response specific to the command is dynamically created bydetermining the characteristics of the command which indicates the topicand/or the action to be performed at the electronic device 100,determining the capabilities of the electronic device 100 on which thecommand needs to be executed. The server 200, based on thecharacteristics of the command and the capabilities of the electronicdevice 100 and/or the server 200, generates the actual response specificto the command and the intelligent response specific to the command.

In an example, the capability may include communication capability in anIoT cloud environment, a location of the server 200 or the electronicdevice 100, a storage capacity, a display capability, a speakercapability, and a vibration capability. In an example, if the user hasan old mobile phone, the display effect for the old mobile phone isdifferent from a new mobile phone. Further, the electronic device 100provides the intelligent response by adjusting audio level based on theuser preferences or any information including the capability of theserver 200 or the electronic device 100. The user preference may beupdated over a period of time. In an example, the intelligent responsemay be related to an old fan, the volume of the old fan is adjustedautomatically.

After generating the intelligent response, the processor 210 maytransmit the intelligent response corresponding to the command to theelectronic device 100 while the electronic device 100 is executing theactual response or before/after the electronic device 100 has executedthe actual response specific to the command.

The processor 210 is configured to execute instructions stored in thememory 230 and to perform various processes. The communicator 220 isconfigured for communicating internally between internal hardwarecomponents and with external devices via one or more networks. Further,the communicator 220 is configured for communicating with the electronicdevice 100.

In an embodiment, the electronic device 100 may generate the actualresponse specific to the command internally, and may obtain theintelligent response from the server 200.

The memory 230 stores and updates the target knowledgebase based on thevarious command executed in the server 200. The target knowledgebasecontains the details about the command output and keeps updating thecommand output over the period of time. The memory 230 stores theintelligent response. The memory 230 stores information about of allkinds of the intelligent response. The memory 230 maintains informationabout resources (e.g., sound files, pattern files or the like)applicable to the particular effect.

The memory 230 stores instructions to be executed by the processor 210.The memory 230 may include non-volatile storage elements. Examples ofsuch non-volatile storage elements may include magnetic hard discs,optical discs, floppy discs, flash memories, or forms of electricallyprogrammable memories (EPROM) or electrically erasable and programmable(EEPROM) memories. In addition, the memory 230 may, in some examples, beconsidered a non-transitory storage medium. The term “non-transitory”may indicate that the storage medium is not embodied in a carrier waveor a propagated signal. However, the term “non-transitory” should not beinterpreted that the memory 230 is non-movable. In some examples, thememory 230 can be configured to store larger amounts of information thanthe memory. In certain examples, a non-transitory storage medium maystore data that can, over time, change (e.g., in random access memory(RAM) or cache).

Although the FIG. 2 illustrates various hardware components of theserver 200 but it is to be understood that other embodiments are notlimited thereon. In other embodiments, the server 200 may include lessor more components. Further, the labels or names of the components areused only for illustrative purpose and does not limit the scope of thedisclosure. One or more components can be combined together to performsame or substantially similar function to provide the surreal responsein the electronic device 100.

FIG. 3 illustrates various elements in the processor 110 of theelectronic device 100, according to an embodiment of the disclosure.

Referring to FIG. 3, the processor 110 may include a virtual assistant310, a voice analyzer 320, a text analyzer 330, an image analyzer 340,and a command executor 350. The virtual assistant 310, the voiceanalyzer 320, the text analyzer 330, the image analyzer 340 and thecommand executor 350 may be implemented as at least one hardwareprocessor.

The virtual assistant 310 may launch a virtual assistance applicationand receive a user input including a command. The voice analyzer 320 mayanalyze a voice command from a user of the electronic device 100. Thevoice analyzer 320 may extract recognizable words and phrases from thevoice command. In an embodiment, if the voice analyzer 320 receivesvoice command of “What is the schedule today?”, the voice analyzer 320may extract the words “what”, “schedule” and “today” from the voicecommand and the command executor 350 may refer to a schedule applicationinstalled in the electronic device 100 based on the analyzed result ofthe voice analyzer 320 to generate an actual response and an intelligentresponse to the voice command.

Likewise, the text analyzer 330 may analyze a text command from the userof the electronic device and the image analyzer 340 may analyze an imageinput made by the user of the electronic device 100 to generate anactual response and an intelligent response to the command.

Although the FIG. 3 shows various hardware components of the processor110 but it is to be understood that other embodiments are not limitedthereon. In other embodiments, the processor 110/210 may include less ormore number of components. Further, the labels or names of thecomponents are used only for illustrative purpose and does not limit thescope of the disclosure. One or more components can be combined togetherto perform same or substantially similar function to provide the surrealresponse in the electronic device 100.

FIG. 4 illustrates a block diagram for providing an intelligent responseto a command, according to an embodiment of the disclosure.

Referring to FIG. 4, the system may include the server 200 communicatedwith the electronic device 100. The electronic device 100 may receivethe command from the user and transmit the command and request of anintelligent response to the server 200. In response to the command andrequest of an intelligent response, the server 200 may analyze thecommand and dynamically generate a feedback specific to the command,wherein the feedback imitate characteristics of the command receivedfrom the electronic device 100.

The electronic device 100 may generate an actual response which isspecific to the command and an intelligent response which includescertain effects corresponding to the command and/or the actual responseby itself. Once the electronic device 100 determines that the actualresponse includes a control signal to control an IoT device 300 based onthe content of the command the electronic device transmits the controlsignal to the IoT device 300. For example, if the electronic device 100receives the command of “Open the door” from a user, the electronicdevice 100 may generate the actual response including a control signalfor opening the door and the intelligent response including soundeffects of “door opening”. The control signal for opening the door maybe transmitted to an IoT device 300 which controls opening and closingthe door.

FIG. 5 is an overview of the system providing an intelligent response toa command, according to an embodiment of the disclosure.

Referring to FIG. 5, the system 1000 includes a set of electronicdevices 100 a-100 n, the server 200, a virtual assistant cloud 508 a, anatural language parser (NLP) cloud 508 b, an effect resource database(DB) 504, an IoT Cloud 506, a display subsystem 510, an audio subsystem512, a vibrator subsystem 514, an effect composer 516, a scene managerclient 518, a device profile DB client 520 and a user preference DBclient 522. All the operations and functions of the electronic device100 a through 100 n and the server 200 are explained in conjunction withthe FIGS. 1 through 4. The effect resource DB 504 comprises the audioeffect, the visual effect, the physical effect, the audio-visual effect,and the audio-visual-physical effect. The IoT cloud 506 manages alloperations among the electronic device 100 a-100 n and the server 200using the virtual assistant cloud 508 a. The effect composer 516 maycommunicate with the display subsystem 510, the audio subsystem 512, andthe vibrator subsystem 514. The effect composer 516 produces the variouseffects (e.g., audio effect, visual effect, physical effect,audio-visual effect, audio-visual-physical effect or the like) using thedisplay subsystem 510, the audio subsystem 512, and the vibratorsubsystem 514. The scene manager client 518 may communicate with thecommand executor 350, the user preference DB client 522 and the deviceprofile DB client 520.

FIG. 6 is a flowchart illustrating a method of providing an intelligentresponse, according to an embodiment of the disclosure.

Referring to FIG. 6, the operations of 602 through 608 may be performedby the processor 110.

At operation 602, the processor 110 of the electronic device may receivea command from a user of the electronic device 100. The command may beany kind of user inputs using the electronic device. That is the commandmay be a voice command, a text command, or a touch input command. Acapturing of an image may be another form of the command.

At operation 604, the processor 110 may analyze the content of thecommand. For example, the voice command of “open the window” isreceived, the processor 110 may control a voice analyzer 320 torecognize and analyze the content of the voice command. The processor110 determines that the user wants to open the window based on theanalysis result and the processor 110 generates an intelligent responsewhich is contextually associated with the content of the command and anactual response to the content of the command at operation 606. Forexample, the processor 110 may generate a sound effect of opening thewindow as the intelligent response based on the analyzed content of thecommand or the processor 110 may generate a moving picture which depictsopening the window displayed on the display 140 of the electronic device100. The actual response may be a control signal transmitted to a windowcontroller and the window controller may open the window based on thecontrol signal.

The intelligent response may be at least one selected from or acombination of the group of a sound effect associated with the contentof the command, a vibration effect associated with the content of thecommand, a still or moving image associated with the content of thecommand, and a visual effect associated with the content of the command.

At operation 608, the electronic device 100 may output the intelligentresponse and the actual response including the control signal asdescribed above.

The various actions, acts, blocks, operations, or the like in the flowdiagram of FIG. 6 may be performed in the order presented, in adifferent order or simultaneously. Further, in some embodiments, some ofthe actions, acts, blocks, operations, or the like may be omitted,added, modified, skipped, or the like without departing from the scopeof the disclosure.

FIG. 7 is a flowchart illustrating another method of generating anintelligent response by the server 200, according to an embodiment ofthe disclosure.

Referring to FIG. 7, the operations 702 through 706 may be performed bythe processor 210.

At operation 702, the server 200 may receive the command from theelectronic device 100. The server 200 may receive a request of anintelligent response along with the command from the electronic device100.

At operation 704, the processor 210 of the server 200 may generate anintelligent response and an actual response specific to the command. Theactual response may contain any information or a control signal whichenables the electronic device 100 to take certain actions correspondingto or in response to the command. In an embodiment, the actual responsemay be generated by the electronic device 100.

At operation 706, the server 200 transmits the intelligent responsegenerated based on the command and the request of the intelligentresponse. The server 200 may obtain information regarding the commandand corresponding intelligent response from a database (DB) connectablevia a network.

The various actions, acts, blocks, operations, or the like in theflowchart of FIG. 7 may be performed in the order presented, in adifferent order or simultaneously. Further, in some embodiments, some ofthe actions, acts, blocks, operations, or the like may be omitted,added, modified, skipped, or the like without departing from the scopeof the disclosure.

FIG. 8 illustrates an example scenario providing an intelligent responseto an inquiry, according to an embodiment of the disclosure.

Referring to FIG. 8, the user may select an object in the imagedisplayed on the electronic device 100. The object appears to be a birdand the user wants to know the name of the bird. Thus, the user inputs avoice command of an inquiry—“What is this bird?” The electronic device100 may receive the voice command including the inquiry and theprocessor 110 may analyze the shape and the features of the image andsearch an internet or a network for the bird having similar image toobtain the name. If the search result indicates that the imagecorresponds to a duck, the electronic device 100 may generate an actualresponse of “the bird is a duck” and an intelligent response of ducktwitting sound. If the electronic device 100 determines that the ducktwitting sound is not available in the memory 130 of the electronicdevice 100, the electronic device 100 may search the internet for theduck twitting sound and obtain the duck twitting sound.

Once the electronic device 100 obtains the duck twitting sound, theelectronic device 100 may output the actual response and the intelligentresponse to the user.

FIG. 9 illustrates an example scenario of providing an intelligentresponse with a sound effect, according to an embodiment of thedisclosure.

Referring to FIG. 9, the user may provide a command of “Close the windowblinds” at operation 901. While executing the command or once thecommand is executed make special intelligent effects using capabilitiesof the electronic device 100, the electronic device 100 creates theintelligent response including at least one of the audio only effect,the audio-visual effect, and the audio-visual-physical effect forclosing the window blind. The intelligent response may mimiccharacteristics of the command being executed or may also include thecharacteristics of the electronic device on which the command is beingexecuted or may recreate the context associated with the actual responseor may be purely imaginary (is not unconditional). Further, the system1000 decides when to create the intelligent response and when not tocreate the intelligent response. In an example, the system 1000 maygenerate an actual response including a control signal for the windowblind closing with the intelligent response having sound effects ofclosing the window blind based on the user presence nearby theelectronic device 100. The user may hear the sound effect of closing thewindow blind while the actual window blind is closed at operations 903and 905.

FIG. 10 illustrates an example scenario of controlling light operations,according to an embodiment of the disclosure.

Referring to FIG. 10, there is an IoT bulb in a room. The user providesthe command of “Turn off the bulb”. Based on the command, the electronicdevice 100 identifies that the command operation is “turning off light”.Further, the electronic device identifies the effect from a database(DB) connectable via an internet. While switching off the bulb, adisplay of the electronic device 100 is controlled to be dimmed to matchthe relative brightness (e.g., 30%) of a bed room.

FIG. 11 illustrates an example illustration in which an intelligentresponse associated with schedules is provided, according to anembodiment of the disclosure.

Referring to FIG. 11, a user of the electronic device 100 may input avoice command as “What are my schedules today?” to a virtual assistanceof the electronic device 100 at operation 1101.

At operation 1103, the electronic device 100 analyzes the contentincluded in the voice command and recognizes that the user would like toknow today's schedule. The electronic device 100 may generate soundeffects of turning over diary or calendar pages to check schedules inthe diary or the calendar. After outputting the generated sound effects,the electronic device 100 may provide a response of “You have a meetingwith Mark at 4 P.M. and then have a dinner with Isabelle at The Rockcafe” at operation 1105. After or when outputting the response ofstating today's schedule as above, the electronic device may outputanother sound effects including café sounds based on the analysis of thecontent of the schedule which includes “dinner at the Rock Café.” Theelectronic device 100 may connect to the server 200 over the IoT cloud506 for the server 200 to generate the intelligent response. That is,the electronic device 100 may transmit to the server request of theintelligent response based on the analyzed content included the commandand receive the intelligent response generated by the server referringto the analyzed content.

FIG. 12 illustrates an example scenario in which an intelligent responseassociated with a door is provided, according to an embodiment of thedisclosure.

Referring to FIG. 12, the user provides a voice command if “Close thegarage door” at operation 1201. At operation 1203, the electronic device100 may generate, based on the analysis of the content of the command,an intelligent response which includes background sound of closing thegarage door. At operation 1205, the electronic device may output acontrol command as an actual response to the user command. The controlcommand may control the garage door to be closed by transmitting a doorclose signal to a remote device such as the garage door controller (notshown). The sound effects of closing the garage door may be maintaineduntil the garage door controller completes the closing of the garagedoor. The remote device may be any of IoT devices according to variousembodiments of the disclosure.

At operation 1207, the electronic device 100 may output a response of“Garage door closed” when the electronic device 100 determines that theclosing the garage door is completed. The determination may be madebased on a response signal received from the garage door controller (notshown). The response of “Garage door closed” may be a voice response, atext response or a combination thereof.

In an embodiment, the electronic device 100 which receives the commandof “closing the garage door” may be a smart speaker, but the electronicdevice 100 cannot decide what effects to be generated. The electronicdevice may be assisted by the server 200. The server 200 may determinethe effect based on the received content of the command generate anappropriate intelligent response including the effect based on thereceived content of the command.

FIG. 13 illustrates an example scenario in which an intelligent responseassociated with weather is provided, according to an embodiment of thedisclosure.

Referring to FIG. 13, the user provides a command of “Good morningvirtual assistant”! at operation 1301. The electronic device 100 mayanalyze the content of the command and determine that the content isrelated to a greeting in the beginning of a day. Then, the electronicdevice may determine that the user needs weather information based onthe analysis of the content of the command. At operation 1305, theelectronic device may provide a response of “Good morning Jane! It israining outside” by referring to a weather application which providesweather information. At operation 1303, at the same time while providingthe response, the electronic device 100 may provide background rainsound generated based on the analysis of the content of the command andthe weather information obtained from the weather application. In anembodiment, the weather information may be obtained from a weatherapplication installed in the server 200.

FIG. 14 illustrates an example scenario in which an intelligent responseassociated with sports event is provided, according to an embodiment ofthe disclosure.

Referring to FIG. 14, at operation 1401, the user provides a voicecommand of “virtual assistant! What is current LA dodgers score?” Theprocessor 110 may analyze the content of the command and extractmeaningful words and phrases such as “LA Dodgers” and “score” afterprocessing the voice command with the voice analyzer 320. The processor110 determines that the user wants to know the baseball score of LADodgers and obtains major league baseball information including the livescore of LA Dodgers as actual response via a network. The major leaguebaseball information includes the current score of LA Dodgers whichshows that LA Dodgers 4: 3 San Francisco Giants. Then the processor 110may generate an intelligent response which includes background soundeffect of baseball stadium noise based on the determination that LADodgers is one of baseball teams at operation 1403. The processor 110may generate, based on the major league baseball information, an actualresponse which includes the information that “LA Dodgers is leading SanFrancisco by 4-3” at operation 1405. The intelligent response may be avoice output via the speaker of the electronic device 100 and the actualresponse may be a voice output via the speaker or a text outputdisplayed on the display 140 of the electronic device 100.

FIG. 15 illustrates an example scenario in which an intelligent responseassociated with light on/off is provided, according to an embodiment ofthe disclosure.

Referring to FIG. 15, the brightness of the display is changed from anormal state 1510 to a dimmed state 1520 in response to a command ofturning off light. The user may provide a voice command of turning offlight. Based on the user's voice command, the display screen becomesdark as an intelligent response generated based on the analysis of theuser's voice command.

FIG. 16 illustrates an example scenario in which an intelligent responseassociated with fan speed is provided, according to an embodiment of thedisclosure.

Referring to FIG. 16, a user provides a command of “increase fan speed”via the virtual assistant of the electronic device 100. In response tothe command, the electronic device 100 may generate a control signal tocontrol a fan device to increase the fan speed. While increasing the fanspeed, a combination of intelligent response may be outputted. Forexample, an animation of fan blowing appears on the display 140 alongwith breeze sound and feeble vibration to imitate increased fan speedusing the augmentable virtual effect.

The degree of vibration, the fan speed illustrated by the animation andthe volume of the breeze sound of the electronic device 100 may increaseproportional to the fan speed.

FIG. 17 illustrates an example scenario in which an intelligent responseassociated with playing a song is provided, according to an embodimentof the disclosure.

Referring to FIG. 17, the user requests the electronic device 100 toplay a song (e.g., Yesterday by Beatles) in background while browsing aninternet. Based on the request, the processor 110 searches the networkor the memory 130 for the song (using a music service available) andstarts playback of the song in the background while browsing theinternet.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that the various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. A method of providing an intelligent response onan electronic device, the method comprising: receiving a command from auser of the electronic device; identifying content of the command basedon analyzing the command; generating, based on the content of thecommand, a control signal corresponding to an actual response responsiveto the command; generating at least one intelligent responsecontextually associated with the content of the command and the actualresponse, the at least one intelligent response including a sound effectassociated with the content of the command, the sound effect includingcontent which mimics a physical action of an action being performedremotely from the electronic device by the control signal correspondingto the actual response; determining, with at least one GPS, a locationof the electronic device relative to a location of at least one remotedevice performing the physical action in response to the control signalfrom the electronic device; and outputting the at least one intelligentresponse including the sound effect at the electronic device and thecontrol signal, wherein the sound effect is maintained until thecompletion of the physical action, and wherein the outputting the soundeffect comprises outputting the sound effect with a first volume basedon a determination that the electronic device is located within a firstdistance from the at least one remote device and outputting the soundeffect with a second volume which is larger than the first volume basedon a determination that the electronic device is located a seconddistance which is relatively longer than the first distance from the atleast one remote device.
 2. The method of claim 1, wherein the commandcomprises at least one of a voice command, a text command, a touch inputcommand or an image input captured by a capturing device of theelectronic device.
 3. The method of claim 1, wherein the outputting ofthe at least one intelligent response and the control signal comprisesoutputting the at least one intelligent response and the control signalin a temporal order or simultaneously.
 4. The method of claim 1, whereinthe generating of the at least one intelligent response comprises:transmitting, to a server, a request of the at least one intelligentresponse and the analyzed command; and receiving the at least oneintelligent response from the server.
 5. The method of claim 1, whereinthe control signal corresponding to the actual response is transmittedto the at least one remote device for controlling, in response to thecommand, the at least one remote device.
 6. The method of claim 1,wherein the generating of the at least one intelligent responsecomprises generating the at least one intelligent response further basedon information associated with the command, and wherein the informationbeing obtained from an application is installed in the electronic deviceor a server connected to the electronic device.
 7. The method of claim1, wherein the at least one intelligent response further includes avisual effect associated with the content of the command, the visualeffect comprising a visual effect of changing a brightness of a displayof the electronic device based on the content of the command.
 8. Themethod of claim 1, wherein the at least one intelligent response furtherincludes a visual effect associated with the content of the command, thevisual effect comprising a text output associated with the content ofthe command.
 9. The method of claim 1, further comprising: moving animage associated with the content of the command, wherein the moving ofthe image comprises moving one or more images depicting the content ofthe command.
 10. The method of claim 1, wherein the analyzing of thecommand comprises obtaining information associated with the command froma database via a network.
 11. The method of claim 1, wherein the atleast one intelligent response further comprises a visual effect, thevisual effect comprising displaying a still or a moving image associatedwith the content of the command.
 12. The method of claim 1, wherein thegenerating of the at least one intelligent response comprises searchinga network for first data required to generate the at least oneintelligent response and generating the at least one intelligentresponse based on the first data.
 13. The method of claim 12, whereinthe generating of the control signal corresponding to the actualresponse comprises searching the network for second data required togenerate the control signal and generating the control signalcorresponding to the actual response based on the second data.
 14. Anapparatus for providing an intelligent response, the apparatuscomprising: a display; a GPS sensor; a speaker; and at least oneprocessor configured to: receive a command from a user, identify contentof the command based on analyzing the command, generate, based on thecontent of the command, a control signal corresponding to an actualresponse responsive to the command, generate at least one intelligentresponse contextually associated with the content of the command and theactual response, the at least one intelligent response including a soundeffect associated with the content of the command the sound effectincluding content which mimics a remote physical action of an actionbeing performed remotely from the apparatus by the control signalcorresponding to the actual response, determine, with the GPS sensor, alocation of the apparatus relative to a location of at least one remotedevice performing the physical action in response to the control signalfrom the apparatus, and output the at least one intelligent responseincluding sound effect at the apparatus and the control signal, whereinthe sound effect is maintained until the completion of the physicalaction, and wherein the outputting the sound effect comprises outputtingthe sound effect with a first volume based on a determination that theapparatus is located within a first distance from the at least oneremote device and outputting the sound effect with a second volume whichis larger than the first volume based on a determination that theapparatus is located a second distance which is relatively longer thanthe first distance from the at least one remote device.
 15. Theapparatus of claim 14, wherein the at least one processor is furtherconfigured to: transmit, to a server, a request of the at least oneintelligent response and the analyzed command; and receive the at leastone intelligent response generated by the server.
 16. The apparatus ofclaim 14, wherein the at least one intelligent response furthercomprises a visual effect associated with the content of the command,the visual effect comprising a visual effect of changing a brightness ofthe display of the apparatus based on the content of the command. 17.The apparatus of claim 16, wherein the at least one processor is furtherconfigured to transmit, to at least one illumination device, a signal tochange the brightness of the at least one illumination deviceproportional to the visual effect of changing the brightness of thedisplay of the apparatus based on the content of the command.
 18. Theapparatus of claim 14, wherein the at least one processor is furtherconfigured to search a network for first data required to generate theat least one intelligent response and generate the intelligent responsebased on the first data.
 19. The apparatus of claim 18, wherein the atleast one processor is further configured to search the network forsecond data required to generate the control signal corresponding to theactual response and generate the control signal based on the seconddata.
 20. A non-transitory computer program product comprising acomputer readable storage medium having a computer readable programstored therein, the computer readable program, when executed on anelectronic device, causing the electronic device to: receive a commandfrom a user of the electronic device; identify content of the commandbased on analyzing the command; generate, based on the content of thecommand, a control signal corresponding to an actual response responsiveto the command; generate at least one intelligent response contextuallyassociated with the content of the command and the actual response, theat least one intelligent response including a sound effect associatedwith the content of the command, the sound effect including contentwhich mimics a physical action of an action being performed remotelyfrom the electronic device by the control signal corresponding to theactual response; determine, with at least one GPS, a location of theelectronic device relative to a location of at least one remote deviceperforming the physical action in response to the control signal fromthe electronic device; and output the at least one intelligent responseincluding the sound effect at the electronic device and the controlsignal, wherein the sound effect is maintained until the completion ofthe physical action, and wherein the outputting the sound effectcomprises outputting the sound effect with a first volume based on adetermination that the electronic device is located within a firstdistance from the at least one remote device and outputting the soundeffect with a second volume which is larger than the first volume basedon a determination that the electronic device is located a seconddistance which is relatively longer than the first distance from the atleast one remote device.